|
Randomized (Block) Coordinate Descent Method is an optimization algorithm popularized by Nesterov (2010) and Richtárik and Takáč (2011). The first analysis of this method, when applied to the problem of minimizing a smooth convex function, was performed by Nesterov (2010). In Nesterov's analysis the method needs to be applied to a quadratic perturbation of the original function with an unknown scaling factor. Richtárik and Takáč (2011) give iteration complexity bounds which do not require this, i.e., the method is applied to the objective function directly. Furthermore, they generalize the setting to the problem of minimizing a composite function, i.e., sum of a smooth convex and a (possibly nonsmooth) convex block-separable function: where is decomposed into blocks of variables/coordinates: and are (simple) convex functions. Example (block decomposition): If and , one may choose and . Example (block-separable regularizers): # # , where and is the standard Euclidean norm. ==Algorithm== Consider the optimization problem : where is a convex and smooth function. Smoothness: By smoothness we mean the following: we assume the gradient of is coordinate-wise Lipschitz continuous with constants . That is, we assume that : for all and , where denotes the partial derivative with respect to variable . Nesterov, and Richtarik and Takac showed that the following algorithm converges to the optimal point: Input: //starting point Output: set x=x_0 for k=1,... do choose coordinate , uniformly at random update endfor; 抄文引用元・出典: フリー百科事典『 ウィキペディア(Wikipedia)』 ■ウィキペディアで「Random coordinate descent」の詳細全文を読む スポンサード リンク
|